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Abstract 

We consider a Markov process on a Riemannian manifold, which solves a stochas- 
tic differential equation in the interior of the manifold and jumps according to a 
deterministic reset map when it reaches the boundary. We derive a partial differ- 
ential equation for the probability density function, involving a non-local boundary 
condition which accounts for the jumping behaviour of the process. This is a gen- 
eralisation of the usual Fokker-Planck-Kolmogorov equation for diffusion processes. 
The result is illustrated with an example in the field of stochastic hybrid systems. 
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1 Introduction 

This paper investigates Kolmogorov's forward equation, also called the Fokker- 
Planck equation (FPE), for a class of Markov processes whose behaviour can 
be described as follows: let M be a n-dimensional smooth 2 manifold with 
boundary <9M and interior M, and let 8 = M U T be the topological space 
obtained by adding to M a finite (possibly empty) set T of isolated points. 

* Corresponding author. 

1 Email adresses: Firstname.Lastname@Supelec.fr 

2 "smooth" stands for C°°, here and throughout the whole paper, even though we 
do not really need that much regularity to prove our results. 
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Our process X evolves in M according to a Stratonovich stochastic differential 
equation 3 (SDE) on M, 



dX t = A (X t )dt + ]T A r .(X t ) o dB r t , (,Y) 

r=l 

where B is a n-dimensional Wiener process, o d denotes the Stratonovich dif- 
ferential, and the A r 's are n + 1 smooth vector fields on M. If X happens to 
hit the boundary <9M, it is instantaneously reset either in M or to one of the 
terminal states x G T, according to a measurable map $ : <9M — > S, that we 
call the reset map. 

Our interest for such processes has been motivated by a class of continuous- 
time Markovian models called Stochastic Hybrid Systems (SHS), introduced 
recently by Hu et al. in [2]. Roughly speaking, a SHS is obtained by replacing 
the ordinary differential equations by SDEs in a deterministic hybrid system. 
SHSs are not, of course, the only way to introduce randomness into the de- 
terministic framework of hybrid dynamical systems. Notably, Davis [3] has 
defined and studied a wide class of non-diffusive hybrid processes. See [4] 
and the references therein for a survey of the litterature on continuous-time 
stochastic hybrid models. 

The link between diffusion processes and the FPE has been known since the 
early days of Markov processes [5]. Let us recall some basic facts concerning the 
case of Stratonovich SDEs on a manifold without boundary. From now on, M is 
endowed with a Riemannian metric (•, •) and we denote by m the Riemannian 
volume measure on M. Assume that <9M = and that the solution of (J?) 
has a density p t with respect to m, for all t > 0. Then, under some additional 
smoothness assumptions, p t solves the partial differential equation (PDE) 

-q[ = L K> (!) 

where 

1 n 

L = A) + ~ E A r (2) 
Z r=l 

is the infinitesimal generator of the process, written in Hormander form 4 , and 
L* denotes its formal adjoint. Introducing the probability current vector 

1 11 

J t = p t A - - E div (PtA r ) A r , (3) 

1 r=l 



3 see e.g. Ikeda-Watanabe [1, Chapters 3 and 5] for the basic definitions. 

4 the notation A 2 r stands for the second order differential operator A r o A r , i.e. in 
coordinates A^ip = ^ • A\. di(A 3 r djip). 
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where div denotes the divergence operator on M, the FPE can be rewritten 
as a local conservation equation [6,7]: 

% + div(J t ) = . (4) 



The point of this paper is to prove that, for a SDE with boundary hitting 
resets, under sufficient assumptions: 

- Equation (4) holds in M \ if, where ff = $ (<9M) n M is the subset of M 
where X can re-enter the manifold after having hit the boundary; this is not 
really surprising, since the process behaves like a diffusion process 5 locally 
in M. 

- The well-known absorbing boundary condition, p t = 0, holds on the subset 
of the boundary where at least one of the vector fields A r , 1 < r < n, is not 
tangential. 

- There is a conservation equation for the probability mass flowing through the 
reset map <E>, which is naturally expressed using the probability current J t . 

This will be stated more precisely in Theorem 5, which is our main result. Such 
a PDE was given by Malhame et al. [8], for an example of one-dimensional SHS 
with two discrete states. Their proof is partly heuristic, especially concerning 
the boundary conditions. Our result extends their work in several directions, 
allowing for multi-dimensional domains and terminal states. 

The paper is organized as follows: in section 2 we give a more precise defini- 
tion of SDEs with boundary hitting resets, together with some of their basic 
properties. In the following section, we state and prove our main result, which 
gives the FPE equation for solutions of SDEs with boundary hitting resets. Fi- 
nally, we illustrate the result with two applications, the first of which extends 
Malhame's one-dimensional example. 



2 The model and its basic properties 

2.1 SDE with boundary hitting resets 

Let (Q, J 7 , (J 7 t)t>o, P) be a filtered space carrying a n-dimensional {Ft)- 
Brownian motion B and (X t ) t>0 a right-continuous S- valued adapted process. 



5 This is not a diffusion process, strictly speaking, since the paths are not continuous. 
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Definition 1 (SDE with boundary hitting resets) We say that X solves 
the SDE with boundary hitting resets (S fi , $) up to time (~ if ( is a positive 
stopping time such that, almost surely, 

(i) for all x £ T and all s < (, 

X s = x W £ [s; C) , X t = x, (5) 

(ii) X is piecewise continuous on [0',C)> i.e. has finitely many discontinu- 
ities on each compact set K C [0; Q, 

(Hi) X solves the SDE (5^) on each interval of continuity I C [0; £ A t*), 

where r* is the first- entrance time of X into T, 
(iv) and for each discontinuity time t £ [0',C)> X has a limit on the left 

X- such that X~ E dM and X T = $(X~). 

Proposition 2 (existence and uniqueness) For each J r o-measurable S- 
valued random variable there is a maximal pathwise unique solution (X, Q 
starting from £. More precisely, we mean that: 

(i) X = £ almost surely. 

(ii) ( is a positive stopping time such that X solves $) up to time (~ . 
(Hi) If (X',(') is another solution, then (' < ( and X t = X' t a.s. on [0;£')- 

Proof. 

Existence. We first establish the existence of a solution using a recursive con- 
struction. More precisely, we construct a sequence of right-continuous adapted 
processes (X J ') J - >1 and a sequence of stopping times (t 7 ) j>1 such that, for all 
./> 1- 

(1) Xq = £ almost surely; 

(2) X- 7 has N 3 d discontinuities, < N 3 d < j, which occur at times T\, . . . , t n j; 

d 

(3) X j solves (y, $) up to time r~; 

(4) if Tj < +oo, then X 3 eventually leaves every compact set of M before 
time Tj. 

Let Z° be the solution 6 of (J?) starting from £, defined on the event {£ £ M} 
up to the explosion time e . Exploding solutions are extended to M + with the 



6 Thanks to the smoothness of the vector fields, equation (y) admits a pathwise 
unique maximal solution on any filtered probability space carrying a n-dimensional 
adapted Brownian motion. This solution is defined on a random interval [0; e), where 
e is a stopping time called the explosion time. 
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value A, where A G T is an isolated point that acts as a cemetery. We set 

e if f G M, 



(6) 

+00 otherwise, 



and 



, Z° if £ G M, 
^ = { (7) 
£ otherwise. 

It is clear that X 1 is an adapted process, continuous on [0;tl), which satisfies 
the conditions 1-4. 



Now we construct X 3+l and r J+ i from X 3 and Tj, for j > 1. We set = X, 
on [0',Tj) and then distinguish between several cases when Tj < +00: 



when X 3 has no limit in M at time Tj , we set Tj+i = Tj and = A for 



- when X 3 has a limit in M at time Tj which belongs to <3> 1 (T) , we set 
Tj+1 = +00 and X/ +1 = $(X^'(rr)) for t > r j; 

- and when X 3 has a limit in M at time t~ which belongs to dM. \ (M), 
we set Tj+i = Tj + tj and Xj +1 = (t — Tj), where Z{ is the solution 
of with respect to the Brownian motion B[ = B Tj+u — B Tj , starting 
from $(X J (r~)) and defined up to the explosion time Cj. Note that & is a 
Brownian motion with respect to the filtration (^ r Tj + u ) M >o. 

It is not difficult to check that X^ +1 satisfies the conditions 1-4. The only 
technical point is to verify that it is a (jF t ) -adapted process: this comes from 
the fact that Z J is progressively measurable with respect to the filtration 
(J r Tj +u)u>o- Everything else is a direct consequence of our construction. Finally, 
setting ( = lim.j_> +00 Tj and X t = lmx^+oo X 3 (t) , we obtain a solution of 
$) up to time (~ , starting from £. 

Pathwise uniqueness. Let (X', (') be another solution starting from £. We use 
that the solution of (5^) with a given initial condition is pathwise unique, to 
deduce by recurrence that X = X' almost surely on the interval [0; Tj A ('), for 
all j > 1, and therefore X = X' almost surely on [0; ( A (') since Tj — > (. Then 
we observe that the solution X which has been constructed in the first part of 
the proof cannot be extended beyond the lifetime (. Indeed, when ( < +00, 
one of the following two situations takes place: 

- if N d — lim.j^ +OQ N 3 d < +00, the solution is exploding in M at time T^ d+1 
and has no limit in M; 

- if — +00, the sequence Tj has an accumulation point at time (~ and 
therefore cannot be extended beyond ( without losing the piecewise conti- 
nuity. 
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This implies that (' < (, which completes the proof. □ 



2.2 Basic properties 



Proposition 2 shows that, in general, the solution of a SDE with boundary 
hitting resets cannot be defined for all t > 0, either because the solution of 
the SDE behaves badly or because the process undergoes an infinite number 
of resets within a finite time 7 . Let (X x , ( x ) be the maximal solution of (J^ , $) 
starting from x G S. We assume from now on that 

(Ai) for all x G S, the solution X x has an infinite lifetime, i.e. ( x = +oo 
almost surely. 

Let (R, TZ, (lZt)t>o) be the canonical filtered space of all right-continuous paths 
uj : t G R + I— > cu(t) G S, with coordinate process X t : uj \— > u(t),t > 0. We set 
Q = Q x R and T = T ®'R. For all x G S, we denote by the probability law 
on (fl, J 7 ) defined as the image of P by the application (Id, X x ). In other words, 
Pa: is the extension of P such that X is the solution of (J^, $) starting from x. 
The space (VL^F) is endowed with the filtration (T t )t>o, where T t = T t ® TZt- 

Let § be the extended state space obtained as the union of the state space § 
and the boundary <9M, i.e. S = §U9M = lul We_ denote by C c fe (M) 
the set of compactly supported functions of class C k on M. For convenience, 
we use the notation C k (S), resp. C*(S), to denote the set of all functions 
if : S — > R whose restriction to M belongs to C fc (M), resp. C k (M). Moreover, 
we extend the vector fields A r , < r < n, setting 

(A rV ) (x) = 0, for all i£l (8) 



As in the proof of Proposition 2, we denote Tj the stopping time corresponding 
to the j th jump, j > 1, with the convention that Tj = +oo if the process has 
less than j jumps. 



This is what people in the hybrid systems community call the Zeno phenomenon 
[9], in reference to Zeno of Elea and his famous paradoxes. 
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Proposition 3 The family {P x , x G 8} is a Markovian system of probability 
measures on (Q, J 7 , (J~t)t>o) with the following properties: 

(i) For all x G S, <p G C 2 (S) and t > 0, it holds P x -almost surely that 
<p{X t ) = <p(x) + f (A oV ) (X s ) ds + j^f (A r <p) (X s ) o dB r s 

+ y: {<po*-<p)(x-) . 



Ti<t 



(ii) Let Ho be any probability measure on §. Then, for all ip G C^(S) and for 



all t > 0, 

E w {<p(X t )} = toitp) + E w || o * (Ly>) (X s ) dsj 



(10) 



where L is given by (2) and E Mo denotes as usual the expectation with 
respect to the probability measure P Mo = f s fio(dx) P x . 

Proof. Using the usual chain rule on the intervals of continuity and a recur- 
rence on k, it is easy to show that, for all k > 1, P^-almost surely, 

rt n rt 



rt "• /■£ 

+ E (^)-^(O) 



on [0;rj). This equation holds in fact on [0; +oo) since Tj — > +oo. Moreover, 
we have that X Tj = &(X~) by Definition l(iv), which establishes (9). 

Now we assume that f is compactly supported. The connection between 
Stratonovich integrals and Ito integrals (see e.g. [1, Theorem V.1.2]) yields 

(A r p) (X 8 ) o dBl = (A r ip) (X s ) dB r s + l - (A r V) (X 8 ) ds, 1 < r < n . 

(12) 

Therefore, equation (9) becomes 

<p(X t ) = <p(x) + /* (Up) (X s ) ds + jr f (A r <p) (X s ) dBl 

J ° r = J ° , \ (13) 

+ £ (ipoQ-tp) (X-) , 

Tj<t 

and equation (10) is obtained by taking expectations and using that the pro- 
cess Mf = Y,i<r<n fo(A r <p)(X a ) dBl i s a martingale (since A r (p is bounded). 
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Proof of the Markov property. We define the semigroup (Pt)t>o, as usual, by 
(P t (p)(x) = K x {(p(X t )}. Then, for all s < t, using equation (9) and the mar- 
tingale property of Mf , 

E x { V (X t )\F s } (14) 
= E x I V {X S ) + f (Dp) (X s ) ds+ £ (V ° $ " <P) (K) I Fs \ (15) 

I S<Tj<t ) 

= (P^.<p)(X.) . (16) 

The last line follows from (10) and the fact that, under the probability 
W x { ■ | Fs}, the process (X s+U ) u > is the maximal solution of (^, <3>) start- 
ing from X s and driven by (B s+U ) u > . Therefore, it has the same distribution 
as (X,B) under F X3 (this is a consequence of the uniqueness in law for (S^), 
using a proof similar to that of Proposition 2). □ 

Remark 4- The same Markov process can also be obtained using the "revival 
theorem" of Ikeda-Nagasawa-Watanabe [10,11]. This can be found with full 
details in [12], in a framework which is much more general than ours. The 
idea is to restart the process X 1 in the proof of Proposition 2, according to 
the deterministic kernel N(u,.) = ^(xi( T -( u ) w ))- We have preferred a more 
explicit pathwise construction, which, we hope, provides a better insight into 
the behaviour of the process. 



3 Fokker-Planck-Kolmogorov's forward equation 

3. 1 Main result of the paper 

Let hq be a probability measure on S, and denote by \i t the probability law of 
X t at time t > under P M0 . We assume that 

(A 2 ) for all t > 0, /i t has a density p t = p(-,t) on M, with respect to the 
Riemannian volume measure m. 

Consequently, we can decompose \i t as 

lit = p t m + J2q(x,t)5 x , (17) 

where q(x,t) = fx t ({x}) = P Mo {X t = x}. 

Assumptions about the boundary and the reset function. Set G = (if). 
Since the boundary is zero-dimensional when n — 1, our assumptions will 
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be slightly different in the one-dimensional case and in the multi-dimensional 
case. First, regardless of the dimension, we assume that 



(A3) $|g is a bijection from G to H 8 . 

In the multi-dimensional case, we further assume that 

(A 4 ) H is a closed and orientable hypersurface 9 , and $|g is a 
C 2 -diffeomorphism from G to H. 

Let a be the surface measure induced by (•, •) on <9M and H, when n > 2. We 
define h = |Jac$| on G, where Jac$ is the Jacobian of $ with respect to the 
Riemannian volume forms on G and H (for any choice of orientation), such 



The same formula holds in the one-dimensional case if we set h — 1 and 
interpret a as the counting measure on <9M U H. 

Smoothness assumptions for the probability law. Finally, we assume that p and 
q are smooth enough, that is: 

(A 5 ) p is of class C 2 ' 1 on (M \ H) x R + . Morevover, for all t > 0, p t is 
continuous on M, and the differential dp t is continuous on M \ H with a 
discontinuity of the first kind on H. 

(A 6 ) For each x G T, 1 1— > q (x, t) is continuously differentiable on M + . 

Note that we have not assumed dp t continuous on M, since it is precisely 
the discontinuities of dp t through H that account for the jumping behaviour 
of X. Assumption A 5 implies that, for all t > 0, the probability current J t 
defined by (3) is a C 1 vector field on M \ if, that has a limit on <9M and a 
discontinuity of the first kind on H. We set J° ut = (J t ,v)on <9M, where v 
is the outward-pointing unit normal, and 



on H, where u 12 = —^21 is the unit normal to H directed from side 1 to side 
2, and jf^ is the value of the discontinuous vector field J t on the side % of H, 
i G {1,2}. The first expression makes it clear that the definition of J t m does 
not depend on the choice of an orientation 10 for H. 

8 This assumption could easily be relaxed in our proof, cf. Remark 8. 

9 embedded smooth submanifold, without boundary and of codimension 1 

10 In fact, H does not even need to be orientable for this to be defined [13]. 



that, for all / G C°(H) 




(18) 



(19) 
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Theorem 5 Under assumptions A1-A6, the law \it of X evolves according 
to the following equations: 

^(x,t) = (L* Pt )(x) onM\H, (20) 

^(x,t) = J g J^da onT, (21) 

where G x = $ _1 ({a?}), for all 16T. Moreover, the following boundary con- 
ditions hold: 

J° ut = hJ?o$ onG, (22) 

p t = on<9M , (23) 

where <9M is the subset of the boundary defined by 

<9M = {x G dM\3r G {l,...,ra}, (A r , v) ^0} . (24) 



Remark 6. <9M can be seen as the non-characteristic part of the boundary 
<9M with respect to the operator L*. Indeed, L* is a second order operator 
with principal symbol 

a 2 (L*) (x, = -\ £ £ K (x) Ai (x) (25) 

r=l i,j=l 

= "2 E (E4(s)&J , (26) 

where the £j's are the coordinates of the covector £ in any local orthonormal 
coframe. Therefore, <9M is characteristic at x if and only if A r is tangent to 
<9M at x, for each r G {1, . . . , n}. 

3.1? Proof of Theorem 5 

The result is obvious when /io(M) = 0, since X is then a constant process. 
When // (M) > 0, we observe that P W {-|X G M } = P^, where n' = 
fj,o/ fj,o(M). Thus, it can be assumed without loss of generality that //o(M) = 1, 
i.e. that q(x, 0) = for all x G T. 

> As a first step, we will prove that, for each test function ip G C^(§) such 
that 

tp o $ = tp on G, (27) 

the following equation holds: 

/ <P (Pt ~ Po) dm = f f Lip Ps dm ds - {l( T *< t ) ip . (28) 

Jm jo Jm 
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Indeed, (27) implies that the jump term (<p o $ — ip) (X~ k ) in equation (10) 
vanishes when the process undergoes a reset into M at time t\~. Hence, there 
is at most one term left in the summation, corresponding to a possible jump 
from <9M to a terminal state in T at time r*: 

E M0 E (v>°*-¥>) (X~) \ = E w {l (T .< t) (<p o $ - ip) (X-)} (29) 

U<i J 

(30) 

Using Fubini's theorem together with the decomposition (17) of \i t and the 
fact that Lip vanishes on T, we can rewrite the second expectation in (10) as 



E w {/ (Lp)(X s ) ds) = [[ (Lp)(X(s,uj))¥, (duj)ds (31) 

= t I Lip dfi s ds (32) 
io is 

= / / L<£> p s dm ds . (33) 
Jo Jm 

Using equation (17) once more, we can also expand the left-hand side of (10): 

W(X t )} = 52<p(x)q(x,t) + / ipptdm. (34) 



Finally, replacing expressions (30), (33), and (34) back into (10) completes the 
proof of equation (28). 

> The next step of the proof relies on the following version of Stokes's formula: 

/ divAdm= / (A,u)da- / ( A® - A™ , v 12 ) da , (35) 
Jm\h JdM Jh \ ' v 

where A is a compactly supported vector field on M, of class C 1 in M \ H, 
extending continuously to <9M and having a discontinuity of the first kind 
on H. This formula is easily proved using the usual divergence theorem [14, 
Theorem 14.34] or the general results of [13, Chapter IX]. Using this formula 
together with the assumption A 5 that p t is continuous on M, for all t > 0, 
yields: 

/ Lpptdm = / ipL*p t dm + (3 t (<p) , (36) 
Jm Jm\h v ' 

where L* is the formal adjoint of L, i.e. 

1 n 

L* Pt = - div (p t A ) + - ]T div ( div iptA r ) A r ) , (37) 

1 r=l 
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and p t is a distribution supported by <9M U H: 
Pt(<p) = \ E / ArVPtiA, v)da+ ( ipJ^ da- f ip j™ da. (38) 

I ~y J 8M J <9M JH 

A detailed derivation of these equations can be found in appendix 5.1. Using 
(36), we rewrite equation (28) as 

/ ippt dm = f f3 s (ip) ds - E M0 {l (r *< t) </? (X~)\ , (39) 
Jm\h Jo l \ / j 

where 

Pt = Pt-Po- L*p s ds (40) 
Jo 

is well-defined and continuous on M \ H. 

> The last step of the proof consists in choosing specific test functions (p in 
equation (39). First, it is easy to check that, if ip has its support in M \ H, 
then (27) is automatically satisfied since <p vanishes on both G and H, and 
equation (39) becomes 

f cpp t dm = 0, (41) 
Jm\h 

which proves that p t — 0, for alH > 0. Equation (20) then follows by differ- 
entiation. 

Before proceeding to the derivation of the three other equations of the the- 
orem, we shall state a useful technical lemma, whose proof can be found in 
appendix 5.2. 

Lemma 7 For any rji G C^(dM), i G {1,2}, such that r)i\ G is compactly 
supported in G, we can find a function ip G C^(S) such that 

(i) ip satisfies (27), 

(ii) ip = rji on <9M, 

(Hi) and & = r] 2 on dM. 

Proof of equation (23). For any rj G C^(dM), Lemma 7 provides us with a 
function ip G C c 2 (§) such that (27) is satisfied, ip = on dM U H and f£ = 77 
on dM.. Then equation (39) implies that (5 t {ip) = 0, for all t > 0. Using 
equation (38) and the fact that ip vanishes on dMUH, this can be written as 

n „ 

E / AtpPt(A r , u) da = 0. (42) 

~± J dM 

Moreover, by construction, ip has the property that A r ip = ( A r , v) rj, for all 
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r G {1, . . . ,n}, which allows to rewrite equation (42) as 

/ (]T (A r , u) 2 ) p tV da = 0. (43) 

JdM V r =i J 

This holds for all r] G C*(dM), which proves that p t (e™=i (A r , u) 2 ) = 

on <9M, for all t > 0. Observing that £™ =1 ( > on <9M establishes 
equation (23). 

Proof of equation (22). For any r] G C%(G), we can find by Lemma 7 a function 
V? G Cc(S) satisfying (27) and 



i] on G, 

onM\ G. 



(44) 



For such a function, equation (39) reduces to 

f ipJ° ut da - [ up J™ da = 0. (45) 
Then, since ip satisfies (27) and ud = r\ on G, we have 

f r] ( J t out - h Jl n o $) da = . (46) 

This holds for all rj G C%(G), which proves equation (22). 

Proof of equation (21). This time we choose rj G C^(G X ) for some x G T, ip = rj 
on G x and y9 = on <9M \ G x . Equation (39) then becomes 

E .o {\r*< t ,x- ec x} V {X-)) = fj^ dads. (47) 

This implies by dominated convergence that 

P M P < *, K* eK) = J* J k J° ut da ds , (48) 

for any compact subset K of G x . Since the left-hand side is increasing with t, 
this shows that J° ut > on G x , for all s > 0. Consequently, letting K j G x , 
equation (48) yields 

P w {r* < t, X- G G x ) = fj G JT dads (49) 

by monotone convergence. Finally, observing that 

^{r*<t, X- EG X } = q(x,t) (50) 
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yields equation (21) and thus completes the proof of Theorem 5. 

Remark 8. This proof can easily be generalized to the case where $ is no longer 
a diffeomorphism, but still a local diffeomorphism such that $ _1 {{x}) is finite 
for all x G H . In this case, we have 

J?(x) = £ h-\ y ) J° t u \y) , VxeH,Vt> 0, (51) 
instead of equation (22). 



4 Examples 

4-1 Stochastic hybrid systems 

As a first example, we will give the FPE for a multi-dimensional SHS with 
two discrete states, which models the temperature in a house with n rooms, 
n > 1, regulated by a single thermostat. This is a generalisation of the one- 
dimensional process that was studied in [8]. 

Let the n-tuple 9 — (9\, . . . , 9 n ) G W 1 describe the temperature in the n rooms 
of the house, and q G Q = {0, 1} the binary state of the thermostat. The global 
state of the system is then described by the variable x — (q,9) G QxR n , which 
has both a discrete and a continuous component, hence the name "hybrid". 
For a given state q G Q of the thermostat, the temperature 9 t evolves in W 1 
according to a SDE of the form 

d& t = f( q ,e t )dt + 7 d£ t , (y q ) 

where 7 G M. nxn and f g = f(q, •) describes the action of the thermostat, the 
effect of the exterior environment, and the coupling between the temperatures 
of adjacent rooms. The switching of the thermostat is controlled by a linear 
criterion ^f(9) = J2i=i a i@i- The thermostat switches on when \l/(©t) crosses 
some threshold ^ m in downwards, and switches off when it crosses another 
threshold \l/ ma x > ^min upwards. This can be described in the SHS framework 
as follows: we define 

M = {^R n |^)>f min } , (52) 
M x = {9eR n \^(9)<y max } , (53) 

and then the so-called "hybrid state space" 

M = {0} x M U {1} x Mi C Qxl n . (54) 



14 



The process <d t evolves continuously in M q according to the SDE ^ as long 
as the thermostat is in state Q t = q, and Q t switches when Q t reaches dM q . 
Therefore, the hybrid process X t = (Q t , t ) takes its values in M. Considering 
the atlas 

A= {({0}xM ,7r),({l}xM 1 ,7r)} , (55) 

where we denote by n the projection n : (q, 9) \— > 9, M can be seen as a smooth 
manifold with boundary, which has two components. The process X t is then 
the solution of a SDE with boundary hitting resets (=5^, <3>), with the reset map 

$ : dM — > M (56) 
(q,9) ^ (l-q,9) (57) 



and the vector fields 



a (x) = ±f(x)4r, 

1=1 



09? 



Ms) = EtJI". l<r<n. (59) 



We note that it is not always possible to view a hybrid state space M = 
U q€ Q{q} x M q as a manifold, since the domains M q have possibly different 
dimensions in the general SHS framework [2]. 

We will now give the FPE equation associated to (J?, <3>) , assuming a priori 
that the assumptions A 1; A 2 and A 5 are satisfied (A 3 and A 4 are easily 
checked, and A 6 is meaningless here since there is no terminal state, i.e. T = 
0). The metric on M is the one induced by the Euclidian metric on each M q , 
q G Q. The density p t can be seen as a pair (pt,pl) of modal densities, where 
Pt is defined on M q . These modal densities solve a system of PDE, 

dpi » a/'rf i » v a 2 rf 

+ 2£ SS' ,eS ' <60) 

where a lJ = Z)"=i7r7r- Assuming that the matrix a is positive, each modal 
density satisfies the absorbing boundary condition p\ = on dM q . Moreover, 
the PDEs are coupled by the conservation equations 

Jr\q, 9) = Jr(l -q,9), q eQ,9edM q , (61) 

where the probability current J t = XT=i ^J" is given by 

1 n dn 

Jl(q,0) = f%e)p t (q,9) - - ^ a« £-(q,9) . (62) 

As in [8], we observe that the drift / does not appear in the conservation 
equations (61), since the absorbing boundary condition holds. 
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4-2 A first exit problem 



Our second application deals with the following problem: let X = (X t )t>o be 
the solution of a SDE of the form (S^) on R n ; given an open subset U C W l 
and an initial probability law /x such that supp /x C U, we want to compute 

n{t) = P M0 {r <t,X T e dUi} , 1 < % < D, < t < T (63) 

where r is the first exit time of X from U, D G N*, T > 0, and {<9£/j, 1 < i < 
.D} is a partition of the boundary dU. We assume that the closure U of U in 
M n is a smooth manifold with boundary, whose interior coincides with U . 

It is well-known [15, Section 5.4] that, if the PDE 
3u ■ 

= Lu t onUx [0;T], (64) 

Ui = on U x {0}, (65) 

^ = \ au . ondUx (0;T], (66) 

has a bounded solution, then 

Ui(x, t) = F x {t < t, X T G dUi] . (67) 

This provides a first approach to our problem, since can be recovered from Ui 
using an integration with respect to /Iq- Another possible approach is provided 
by Theorem 5: we introduce a set of isolated terminal states T = {1, . . . , D} 
and consider the process X which coincides with X up to time r~ and then 
goes to the state i G T such that X~ G dUi. This new process X is the solution 
of a SDE with boundary hitting resets {5?, $) on U U T, where the reset map 
$ is defined by <&(x) = % for all x G dUi, 1 < i < D. The functions can be 
interpreted in this framework as 

n (t) = F^{x t = i} = q(i,t). (68) 

Assuming that the density exists and is smooth enough, the r^'s can be ob- 
tained according to Theorem 5 by solving 

^ = L*p onUx [0;T], (69) 

p = on dU* x [0;T], (70) 

^ = L on[0;T]. (71) 

where dU* is defined as <9M in Theorem 5. We point out that our approach 
based on the forward equation requires the resolution of a single PDE, in con- 
trast with the first approach, based on the backward equation, which involves 
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the resolution of D PDEs. The drawback is that, using the forward approach, 
the solution is only obtained for the given initial distribution /io- 

Remark 9. 

a) From a practical point of view, such a method is of course limited to pro- 
cesses with a state space of low dimension, where the numerical resolution 
of the PDEs is feasible. 

b) A similar methodology can be used to tackle the problem of reachability 
analysis for stochastic hybrid systems (see [16] for further details), at least 
when the target set is a closed subset with smooth boundary. 



5 Appendices 



5.1 Detailed derivation of equation (36) 



In this appendix we give a more detailed derivation of equation (36). First, 
we choose A = A r (pp t A r in (35). Using that A is continuous on M and that 
div(A) = ptAfy + A r ip div(p t A r ), we deduce that 

/ A 2 (pp t dm + / A r (p div(p t A r ) dm = / A r (pp t (A r ,u)da. (72) 
Jm\h JM\H JdM 

This allows us to rewrite the left-hand side of (36) as 

/ L(pp t dm = [ A (pp t dm + ^ V / A 2 T ipp t dm (73) 
Jm Jm\h 2 Jm\h 

= [ \A QV p t -]-Y.A rV dw(p t A r )\ dm (74) 
Jm\h \ 2 [ / 

1 n r 

+ -Y1 / A r <fPt(A r , u)da (75) 

2 Jem 

r 1 n r 

= / J t y?dm + -V/ A r ipp t (A r , v) do (76) 
Jm\h 2 ^ Jam 

We invoke the divergence formula (35) once more, with A = ipj t , to deduce 
that 

/ J t ipdm + f ipdivJ t dm= f if J° ut da - f <p jl n da . (77) 

JM\H JM\H JdM JH 

Finally, combining equations (76)-(77) and using that div J t = —L*p t yields 
the result. 
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5.2 Proof of the extension lemma 



Let % = {U a , a G A} be the set of all precompact coordinate domains U a , 
with coordinate map *f? a : x G U a h- > (x 1 , . . . , x n ) G K", such that: 

a) either U a n H = or U a H <9M = 0; 

b) if C/ Q n <9M 7^ 0, there is £ > such that 

* a (U a ) = [0;e) x (-eje)"" 1 , (78) 
* Q (C/ Q n c>M) = {0} x (- e; e)"- 1 , (79) 

and g|r is the inward-pointing unit normal vector to <9M, i.e. ^ = — ^fr; 

c) if U a fl if 7^ 0, there is e > such that 

*«(£/«) = (-^) n , (so) 

* Q (C/ a n 9M) = {0} x (- e; e) n - 1 . (81) 

Any x G M has a precompact coordinate neighbourhood satisfying either (b) 
or (c). It can be shrinked to satisfy (a) since if is a closed subset of M according 
to assumption A 4 . Therefore, % is an open cover of M. 

Let {g a , a G A} be a partition of unity subordinate to % . For each a G A, 
we construct a function </? a on ?7 Q as follows: 

- If supp g a intersects neither <9M nor if, we set f a = 0; 

- If suppg Q fl <9M ^ 0, we define ip a in coordinates 11 by 



(Paix 1 ,^) = g a (x l ,x') 



(0, xO-x 1 7/2(0, , (82) 



where x' = (x 2 , . . . ,x n ); 
- If supp g a n <9M 7^ 0, we set 



</? a (x\x') = Sa (i 1 ) j/)^o$- 1 )(0 I i'). (83) 

These definitions are compatible since, according to (a), U a cannot intersect 
both <9M and H . ip a is of class C 2 since the r/^s are C 2 by hypothesis and <f> 
is a C 2 -diffeomorphism by A 4 . Moreover, ip a is compactly supported in U a 
because supp ip a C suppg a C C/ Q . We extend y2 a to § by ip a = outside U a . 
We observe that ip a 7^ if and only if supp <? Q intersects either supp 771 U supp r] 2 
or $(supp77i fl G), which are both compact sets. Consequently, { a \ ip a 7^ } 
is a finite set because {supp g a , a G A} is locally finite. 

Finally, we set ip = J^aVa- This is in fact a finite sum by the preceding ob- 
servation, which implies that <f> G C 2 (E>). The properties (i)-(iii) of the lemma 



11 We identify a point x € M with its coordinates (x 1 , . . . , x n ) in the chart (U a , ^ c 
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follow then easily from our construction, using that J2 a 9a — 1, and the proof 
of Lemma 7 is complete. 
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